[Python] Fix AttributeError in ExternalTransform.expand by using get_type_hints() #37299

Eliaaazzz · 2026-01-14T03:29:06Z

Description

Fixes #37289 ： Replaces direct access to the internal _type_hints attribute with the public get_type_hints() accessor in ExternalTransform.expand().

Why this is needed

Directly accessing self._type_hints causes an AttributeError if the type hints have not been explicitly initialized before the expansion phase. This issue was observed specifically when using StorageWriteToBigQuery with with_auto_sharding=True, which triggers an ExternalTransform expansion where _type_hints is missing.

Using get_type_hints() is the correct, defensive approach as it ensures valid type hints (or an empty instance) are returned, preventing the crash.

Testing

Reproduced the crash locally using a script involving StorageWriteToBigQuery with auto-sharding enabled.
Verified that the fix resolves the AttributeError and allows the pipeline to proceed past the expansion phase.

Checklist

Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
Update CHANGES.md with noteworthy changes.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

gemini-code-assist · 2026-01-14T03:29:20Z

Summary of Changes

Hello @Eliaaazzz, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves a critical AttributeError that could arise during the expansion phase of ExternalTransform when the internal _type_hints attribute was accessed before its initialization. By adopting the more robust get_type_hints() method, the system now defensively retrieves type hints, ensuring pipeline stability and preventing crashes in scenarios where type hints might be absent or not yet set up, thereby improving the reliability of data processing operations.

Highlights

AttributeError Fix: Replaced direct access to the internal self._type_hints attribute with the public self.get_type_hints() method within ExternalTransform.expand(). This change prevents an AttributeError that occurred when _type_hints was not explicitly initialized, particularly impacting StorageWriteToBigQuery with auto-sharding.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

codecov · 2026-01-14T03:59:45Z

Codecov Report

❌ Patch coverage is 80.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 55.17%. Comparing base (4dcc27d) to head (8fe6cc6).

Files with missing lines	Patch %	Lines
sdks/python/apache_beam/transforms/external.py	80.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #37299      +/-   ##
============================================
+ Coverage     55.16%   55.17%   +0.01%     
  Complexity     1676     1676              
============================================
  Files          1068     1068              
  Lines        167257   167342      +85     
  Branches       1208     1208              
============================================
+ Hits          92261    92338      +77     
- Misses        72816    72824       +8     
  Partials       2180     2180

Flag	Coverage Δ
python	`80.79% <80.00%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions · 2026-01-14T04:48:54Z

Assigning reviewers:

R: @jrmccluskey for label python.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

…type_hints() (Fixes apache#37289)

Eliaaazzz · 2026-01-14T13:13:27Z

I have pushed the fix for the linting issue.

Regarding the Python 3.13 failure, it seems unrelated to my changes since the tests for Python 3.10, 3.11, and 3.12 all passed. I also noticed the logs indicate '272 similar failures in the previous 7 days', suggesting this might be a known flaky test in the CI environment.

Could you please take a look?

Eliaaazzz · 2026-01-14T14:19:21Z

Thanks @derrickaw! I've fixed the line length issue in the latest commit. Standing by for the next steps.

derrickaw · 2026-01-14T15:46:04Z

Looks like a few more lint issues that need resolving. Make sure to have gone through this to assist - https://github.com/apache/beam/blob/master/contributor-docs/python-tips.md#lint-and-formatting-checks

Eliaaazzz · 2026-01-15T07:38:25Z

Looks like a few more lint issues that need resolving. Make sure to have gone through this to assist - https://github.com/apache/beam/blob/master/contributor-docs/python-tips.md#lint-and-formatting-checks

Hi Derrick, the CI checks are all green now. The previous failure appeared to be a flaky test (infrastructure issue with Socket closed), and it passed successfully after a retry.

Please let me know if there is anything else needed. Thanks!

derrickaw · 2026-01-15T13:02:49Z

LGTM - will let @jrmccluskey do final review. Thanks.

Eliaaazzz · 2026-01-15T13:24:41Z

LGTM - will let @jrmccluskey do final review. Thanks.

Thanks for your patience and guidance, appreciate it!

jrmccluskey

LGTM. Good catch, thanks!

Eliaaazzz · 2026-01-15T23:44:18Z

LGTM. Good catch, thanks!
Thank you for the quick review and for merging this! Glad I could help improve the robustness here. Looking forward to more contributions.

github-actions bot added the python label Jan 14, 2026

github-actions bot added the Next Action: Reviewers label Jan 14, 2026

Eliaaazzz force-pushed the users/Eliaaazzz/fix-issue-37289 branch from 8fe6cc6 to 69f022f Compare January 14, 2026 05:12

[Python] Fix AttributeError in ExternalTransform.expand by using get_…

e74143d

…type_hints() (Fixes apache#37289)

Eliaaazzz force-pushed the users/Eliaaazzz/fix-issue-37289 branch from 69f022f to e74143d Compare January 14, 2026 07:49

Fix lint: wrap long comment at line 808

887d603

github-actions bot added build and removed build labels Jan 15, 2026

Fix yapf formatting and ignore local venv

7510d50

Eliaaazzz force-pushed the users/Eliaaazzz/fix-issue-37289 branch from 33e8676 to 7510d50 Compare January 15, 2026 06:25

github-actions bot added build and removed build labels Jan 15, 2026

jrmccluskey approved these changes Jan 15, 2026

View reviewed changes

jrmccluskey merged commit 68e0d66 into apache:master Jan 15, 2026
101 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Python] Fix AttributeError in ExternalTransform.expand by using get_type_hints() #37299

[Python] Fix AttributeError in ExternalTransform.expand by using get_type_hints() #37299

Eliaaazzz commented Jan 14, 2026 •

edited

Loading

Uh oh!

gemini-code-assist bot commented Jan 14, 2026

Uh oh!

codecov bot commented Jan 14, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Jan 14, 2026

Uh oh!

Eliaaazzz commented Jan 14, 2026

Uh oh!

Eliaaazzz commented Jan 14, 2026

Uh oh!

derrickaw commented Jan 14, 2026

Uh oh!

Eliaaazzz commented Jan 15, 2026

Uh oh!

derrickaw commented Jan 15, 2026

Uh oh!

Eliaaazzz commented Jan 15, 2026

Uh oh!

jrmccluskey left a comment

Uh oh!

Uh oh!

Eliaaazzz commented Jan 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[Python] Fix AttributeError in ExternalTransform.expand by using get_type_hints() #37299

[Python] Fix AttributeError in ExternalTransform.expand by using get_type_hints() #37299

Conversation

Eliaaazzz commented Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Why this is needed

Testing

Checklist

Uh oh!

gemini-code-assist bot commented Jan 14, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

codecov bot commented Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Jan 14, 2026

Uh oh!

Eliaaazzz commented Jan 14, 2026

Uh oh!

Eliaaazzz commented Jan 14, 2026

Uh oh!

derrickaw commented Jan 14, 2026

Uh oh!

Eliaaazzz commented Jan 15, 2026

Uh oh!

derrickaw commented Jan 15, 2026

Uh oh!

Eliaaazzz commented Jan 15, 2026

Uh oh!

jrmccluskey left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Eliaaazzz commented Jan 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Eliaaazzz commented Jan 14, 2026 •

edited

Loading

codecov bot commented Jan 14, 2026 •

edited

Loading